Skip to content

fix!: pass variadic inputs in connection order in Pipeline.run() and run_async() - #13415

Merged
davidsbatista merged 11 commits into
mainfrom
fix/async-run-lazy-variadic-order
Nov 9, 2026
Merged

davidsbatista merged 11 commits into
mainfrom
fix/async-run-lazy-variadic-order

Conversation

@davidsbatista

@davidsbatista davidsbatista commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Related Issues

Proposed Changes:

The problem

A component with a lazy variadic input (DocumentJoiner.documents, AnswerJoiner.answers, any Variadic[...] or multi-sender list[...] input) received its values in arrival order:

  • In run(), that's the execution order. It's deterministic, but hard to predict: it depends on depth, deferral of components with optional inputs, and tie-breaks.
  • In run_async(), it's completion order, so it depends on timing.

So run_async() could give different results between runs, platforms and Python versions, and different results from run(). For example, DocumentJoiner(join_mode="merge", weights=[0.7, 0.3]) could apply the weights to the wrong retrievers.

The fix

Instead of depending on when each sender runs or finishes, the order now comes from the pipeline definition: a lazy variadic input receives its values in the order its senders were connected with Pipeline.connect(). Each input socket already records its senders in that order, so the pipeline sorts the values by it just before the component reads them.

The same rule applies in run(), run_async() and run_async_generator(), so the order is the same in every run and in both modes. Users control it through their connect() calls: weights[i] of a DocumentJoiner applies to the i-th connection. GreedyVariadic (BranchJoiner) is unchanged, since it takes the first value that arrives.

Other options explored (all async only, trying to match run()'s order)

  1. Sort by sender name. Doesn't match run(), where deeper senders run later whatever their name. It fails the hybrid retrieval test.
  2. Sort by depth, then name. Misses run()'s deferral of components with unfilled optional inputs, and swaps the joiner weights when an optional filter branch doesn't fire.
  3. Replay run()'s scheduler during the async run. Exact, but it means a second scheduler of a few hundred lines to keep in sync with run(), and joiners can start later.

run()'s execution order is hard to reproduce and hard for users to predict, so this PR uses connection order instead.

Code changes (haystack/core/pipeline/base.py)

  • _consume_component_inputs() sorts each lazy variadic socket's entries with _sort_by_connection_order() before flattening them. Both run modes go through this function.
  • _connection_index() gives each entry's sort key:
    • -1 for user inputs;
    • the sender's position in socket.senders, which connect() already maintains;
    • len(socket.senders) for unlisted senders, which go last.
      The sort is stable.
    • to_dict() lists the connections into each socket in connection order. graph.edges lists them grouped by sender, which lost that order on a round trip. Sockets with a single sender are unaffected.

How did you test it?

New tests, all of them asserting the same result for run() and run_async():

  • test_run_async_passes_variadic_inputs_in_same_order_as_run: a custom Variadic[str] collector, where the sender connected first finishes last in async.
  • test_run_async_applies_document_joiner_weights_to_the_retrievers_in_connection_order: hybrid retrieval with weights, in both connection orders. In each order, one run mode finishes the retrievers in the other order. The expected output is DocumentJoiner.run() called directly on the retriever outputs, in connection order.
  • test_run_async_applies_document_joiner_weights_like_run_when_an_optional_filter_branch_is_skipped: the optional filter case where Option 2 failed.
  • test_run_async_passes_variadic_inputs_in_connection_order_on_every_loop_iteration: connection order holds in every round of a loop, where neither mode's natural order matches it.
  • test_to_dict_from_dict_round_trip_keeps_variadic_connection_order: senders connected in the opposite order to add_component().
  • A new test__consume_component_inputs case: interleaved values from the same sender keep their own order.

Updated tests:

Notes for the reviewer

  • to_dict() output changes order for multi-sender sockets only.

  • run() behaviour changes for pipelines where execution order differs from connection order. For example, which retriever gets which joiner weight, or which copy of a duplicate DocumentJoiner keeps in concatenate mode. The release note is under fixes and tells users to connect senders in the order they want. Should this be fix!: with an upgrade note instead?

@vercel

vercel Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
haystack-docs Ready Ready Preview Nov 6, 2026 3:12pm UTC

Request Review

@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Coverage report

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  haystack/components/retrievers/in_memory
  embedding_retriever.py
  haystack/core/pipeline
  base.py
  pipeline.py
Project Total  

This report was generated by python-coverage-comment-action

@davidsbatista davidsbatista changed the title fix: async run lazy variadic order fix: pass variadic inputs in connection order in Pipeline.run() and run_async() Oct 26, 2026
@julian-risch
julian-risch removed their request for review November 4, 2026 11:51
@anakin87
anakin87 removed the request for review from a team November 4, 2026 15:23
Comment thread haystack/core/pipeline/base.py Resolved
@sjrl

sjrl commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

Now that the order is defined, lets document it in the weights param of DocumentJoiner (haystack/components/joiners/document_joiner.py) and in docs-website/docs/pipeline-components/joiners/documentjoiner.mdx, e.g. "weights[i] applies to the i-th connection made to documents".

@sjrl

sjrl commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

Since we now have an upgrade note, could we change the PR title to fix!:?

@sjrl sjrl left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! One minor comment about updating the PR title

@davidsbatista davidsbatista changed the title fix: pass variadic inputs in connection order in Pipeline.run() and run_async() fix!: pass variadic inputs in connection order in Pipeline.run() and run_async() Nov 9, 2026
@davidsbatista
davidsbatista merged commit 4ca0613 into main Nov 9, 2026
29 checks passed
@davidsbatista
davidsbatista deleted the fix/async-run-lazy-variadic-order branch November 9, 2026 08:57

This branch was successfully deployed

1 active deployment
Preview — b44c8bb1 Deployed Nov 6, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

P1 High priority, add to the next sprint topic:core topic:tests type:documentation Improvements on the docs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Pipeline.run_async passes variadic inputs in completion order, not in a stable order

2 participants